Papers with representation learning
Towards Automated Error Discovery: A Study in Conversational AI (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that LLMs require information about the nature of an error or hints about its occurrence for accurate detection. |
| Approach: | They propose an encoder-based approach to detect and define errors in conversational AI. |
| Outcome: | The proposed framework outperforms baselines across multiple error-annotated dialogue datasets and shows strong generalization to unknown intent detection. |
Unsupervised Multimodal Clustering for Semantics Discovery in Multimodal Utterances (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for semantics discovery focus on text, video, and audio, failing to leverage the rich multimodal information in the real world. |
| Approach: | They propose a method to construct augmentation views for multimodal data and use them to perform pre-training to establish well-initialized representations for subsequent clustering. |
| Outcome: | The proposed method improves on benchmark multimodal intent and dialogue act datasets by 2-6% over state-of-the-art methods. |
Cross-Task Instance Representation Interactions and Label Dependencies for Joint Information Extraction with Graph Convolutional Networks (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on information extraction (IE) has solved the four main tasks separately, thus failing to benefit from inter-dependencies between tasks. |
| Approach: | They propose a model to solve four IE tasks in a single model that captures inter-dependencies between tasks. |
| Outcome: | The proposed model achieves state-of-the-art performance on monolingual and multilingual learning settings with three different languages. |
Neural News Recommendation with Collaborative News Encoding and Structural User Encoding (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing news recommendation models encode news title and content separately without leveraging the structural correlation of user browsing histories to reflect user interests explicitly. |
| Approach: | They propose a news recommendation framework consisting of collaborative news encoding and structural user encode to enhance news and user representation learning. |
| Outcome: | The proposed framework improves the performance of news recommendation on the MIND dataset. |
Batch-Softmax Contrastive Loss for Pairwise Sentence Scoring Tasks (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in machine learning have led to the use of contrastive loss for representation learning. |
| Approach: | They propose to use batch-softmax contrastive loss to train pairwise sentence embeddings . they propose to take a batch-softermax contrastitive loss and train it with different loss functions . |
| Outcome: | The proposed model improves on a number of datasets and pairwise sentence scoring tasks. |
Controlled Text Generation Using Dictionary Prior in Variational Autoencoders (2022.findings-acl)
Copied to clipboard
| Challenge: | Variational autoencoders (VAEs) have been widely applied in text generation tasks, but they suffer from insufficient representation capacity and poor controllability. |
| Approach: | They propose a data-driven prior that has expressivity and controllability. |
| Outcome: | The proposed prior enjoys expressivity and controllability and can be used in language modeling and controlled text generation. |
HiTRANS: A Hierarchical Transformer Network for Nested Named Entity Recognition (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies for named entity recognition focus on flat NER, i.e., without nested entities, by sequence labeling methods. |
| Approach: | They propose a Hierarchical Transformer network which decomposes the input sentence into multi-grained spans and enhances the representation learning in a hierarchical manner. |
| Outcome: | The proposed method achieves much better performance than the state-of-the-art approaches on GENIA, ACE-2004, ace-2005 and NNE datasets. |
METNet: A Mutual Enhanced Transformation Network for Aspect-based Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for learning complex sentences with multiple aspects are ill-equipped to learn complex sentences . |
| Approach: | They propose a mutual enhanced transformation network for the ABSA task . it improves representation learning of the aspect with contextual semantic features . |
| Outcome: | The proposed model improves representation learning of the aspect with contextual semantic features, giving the aspect more abundant information. |
Domain-Specific Word Embeddings with Structure Prediction (2023.tacl-1)
Copied to clipboard
| Challenge: | Current word embedding methods do not provide a way to use or predict information on structure between sub-corpora, time or domain. |
| Approach: | They propose a word embedding method that provides general word representations for the whole corpus, domain-specific representations and embeddable alignment simultaneously. |
| Outcome: | The proposed method provides better performance than baselines on a dataset of science and philosophy articles. |
Comprehensive Multi-Dataset Evaluation of Reading Comprehension (D19-58)
Copied to clipboard
| Challenge: | Recent research aims to facilitate training and evaluation on several reading comprehension datasets at the same time. |
| Approach: | They propose an evaluation server that reports performance on seven diverse reading comprehension datasets and includes synthetic augmentations to test models' ability to handle out-of-domain questions. |
| Outcome: | The evaluation server performs on seven reading comprehension datasets, and collects and includes synthetic augmentations for these datasets to test models' ability to handle out-of-domain questions. |
New Intent Discovery with Pre-training and Contrastive Learning (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for identifying intents from unlabeled utterances are label-intensive, inefficient, and inaccurate. |
| Approach: | They propose a multi-task strategy to leverage unlabeled data and external labeled data for representation learning. |
| Outcome: | The proposed method outperforms state-of-the-art methods on three intent recognition benchmarks. |
Cross-Domain Sentiment Classification using Semantic Representation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on cross-domain sentiment classification ignore the semantic relevance between domains. |
| Approach: | They propose to use Abstract Meaning Representation to help with cross-domain sentiment classification by combining sentence-level AMRs with text-graph interaction models. |
| Outcome: | The proposed model is effective over strong baselines and shows its importance over strong models. |
Box Embeddings: An open-source library for representation learning using geometric structures (2021.emnlp-demo)
Copied to clipboard
Tejas Chheda, Purujit Goyal, Trang Tran, Dhruvesh Patel, Michael Boratko, Shib Sankar Dasgupta, Andrew McCallum
| Challenge: | Recent studies have explored alternative vector representations with different inductive biases or capabilities. |
| Approach: | They propose a Python library that extends probabilistic box embeddings to geometric shapes and regions. |
| Outcome: | The proposed library is fully open source and compatible with PyTorch and TensorFlow. |
Joint Learning from Labeled and Unlabeled Data for Information Retrieval (C18-1)
Copied to clipboard
| Challenge: | Recent studies have focused on neural information retrieval (IR) models. |
| Approach: | They propose a framework which can benefit from both labeled and more abundant unlabeled data . they propose supervised retrieval over several strong baselines for IR . |
| Outcome: | The proposed framework can benefit from labeled and more abundant unlabeled data for representation learning in the context of IR. |
Logographic Information Aids Learning Better Representations for Natural Language Inference (2022.findings-aacl)
Copied to clipboard
| Challenge: | Statistical language models often ignore information related to logographic features of written text . many languages, including Chinese and Vietnamese, use logographic writing systems . |
| Approach: | They propose to provide language models with logographic information to learn better semantic representations. |
| Outcome: | The proposed model improves semantic representations in six languages with logographic writing systems. |
A Representation Learning Approach to Animal Biodiversity Conservation (2020.coling-main)
Copied to clipboard
| Challenge: | Using vector embeddings, we can predict the conservation status of animal species using a machine learning model. |
| Approach: | They propose a method for predicting the conservation status of animal species using custom generated name embeddings from Wikipedia text and animal taxonomy data. |
| Outcome: | The proposed model predicts the conservation status of data deficient species and steers more focus towards them for protection. |
WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discourse (D18-1)
Copied to clipboard
| Challenge: | a corpus of 43 million atomic edits is available for Wikipedia edit history . edits are instances in which a human editor has inserted a single contiguous phrase into, or deleted a contigous phrase from, an existing sentence. |
| Approach: | They use Wikipedia edit history to mine atomic edits across 8 languages . they find edits contain instances in which a human editor has inserted a single phrase into, or deleted a contiguous phrase from, an existing sentence. |
| Outcome: | The data show that edits differ from the language observed in standard corpora and that models trained on edits encode different aspects of semantics and discourse than models trained in raw text. |
Foundation Model for Biomedical Graphs: Integrating Knowledge Graphs and Protein Structures to Large Language Models (2024.acl-srw)
Copied to clipboard
| Challenge: | Transformer model has been a de-facto standard in natural language processing, but it is limited to images, text, and/or sequence data. |
| Approach: | They propose to use a multimodal large language model architecture to handle biomedical graphs such as protein structure and chemical molecules to improve its performance. |
| Outcome: | The proposed architecture can handle multiple data types for biomedical graphs such as protein structure and chemical molecules. |
Privacy-Preserving Graph Convolutional Networks for Text Classification (2022.lrec-1)
Copied to clipboard
| Challenge: | Graph convolutional networks (GCNs) are powerful for representation learning on documents that naturally occur as graphs, but sensitive personal information is prone to privacy leaks. |
| Approach: | They propose a method that adapts differentially-private gradient-based training to GCNs and conduct experiments using two optimizers on five NLP datasets in two languages. |
| Outcome: | The proposed method improves baseline privacy bounds by 2.7 while retaining competitive F1 scores while providing strong privacy guarantees. |
Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on multimodal representation learning has focused on uni-modality pre-training or cross-modalities integration. |
| Approach: | They propose a framework for multimodal representation learning that uses uni-modal contrastive coding and an efficient unimodal feature augmentation strategy to capture intermodal dynamics. |
| Outcome: | The proposed framework surpasses state-of-the-art methods on two public datasets. |
Classifying Argumentative Relations Using Logical Mechanisms and Argumentation Schemes (2021.tacl-1)
Copied to clipboard
| Challenge: | Recent studies have focused on training complex neural networks on labeled data. |
| Approach: | They propose to use logical mechanisms to classify argumentative relations without training on labeled data. |
| Outcome: | The proposed method classifies argumentative relations without training on labeled data significantly better than unsupervised baselines. |
Video-Text Retrieval by Supervised Sparse Multi-Grained Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in video-text retrieval have led to improved representation learning methods. |
| Approach: | They propose a multi-grained sparse learning framework to learn an aligned sparsen space shared between video and text for video-text retrieval. |
| Outcome: | The proposed framework is superior to existing methods on video-text retrieval benchmarks. |
MIReAD: Simple Method for Learning High-quality Representations from Scientific Documents (2023.acl-short)
Copied to clipboard
| Challenge: | Pretrained language models can learn rich textual representations, but they cannot provide powerful document-level representations for scientific articles. |
| Approach: | They propose a transformer-based method that learns semantically meaningful representations from scientific papers by fine-tuning transformer models to predict the target journal class based on the abstract. |
| Outcome: | The proposed method outperforms six existing models for representation learning on scientific documents across four evaluation standards. |
On learning and representing social meaning in NLP: a sociolinguistic perspective (2021.naacl-main)
Copied to clipboard
| Challenge: | linguistic variation allows for the expression of social meaning, information about the social background and identity of the language user. |
| Approach: | They introduce the concept of social meaning to NLP and discuss how sociolinguistics can inform work on representation learning in NLP. |
| Outcome: | The proposed model can be used to learn social meaning in NLP and identify key challenges. |
Pairwise Prompt-Based Tuning with Parameter Efficient Fast Adaptation for Generalized Zero-Shot Intent Detection (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to generalize from seen intents to unseen intents are not effective . Xian et al., 2019: a novel approach to generalized zero-shot intent detection is needed . |
| Approach: | They propose a pairwise prompt-based tuning model with parameter efficient fast adaptation . they leverage hybrid contrastive learning in discriminant space and masked language modeling . |
| Outcome: | The proposed model can generalize to unseen intents with the help of seen intents . the proposed model is based on a pairwise prompt-based tuning model with fast adaptation . |
Few-shot Classification with Hypersphere Modeling of Prototypes (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fewshot learning use embeddings in space, but they lack expressivity and are difficult to perform statistically. |
| Approach: | They propose a method where class information is represented by hyperspheres with dynamic sizes with two sets of learnable parameters: the hypersphere’s center and the radius. |
| Outcome: | The proposed method is much more expressive than embeddings and performs better than statistical modeling. |
Decoder-Only LLMs can be Masked Auto-Encoders (2025.acl-short)
Copied to clipboard
Dan Qiao, Yuan Gao, Zheming Yang, Di Yang, Ziheng Wu, Pengcheng Lu, Minghui Qiu, Juntao Li, Min Zhang
| Challenge: | Modern NLP workflows require different models for generation and embedding tasks. |
| Approach: | They propose a method that transforms an LLM into a Uni-Directional Masked Auto-Encoder. |
| Outcome: | The proposed method achieves state-of-the-art under unsupervised conditions with merely 100 training steps. |
Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretraining (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models learn non-trivial abstractions during pretraining, but it is not well understood when and how these specific linguistic abilities emerge. |
| Approach: | They propose a method to track the evolution of linguistic features during pretraining by using sparse crosscoders to discover and align features across model checkpoints. |
| Outcome: | The proposed approach can detect features emergence, maintenance, and discontinuation during training stages. |
Sequence-to-Action: End-to-End Semantic Graph Generation for Semantic Parsing (P18-1)
Copied to clipboard
| Challenge: | Existing methods for semantic parsing are difficult to design and learn, especially in wideopen domains. |
| Approach: | They propose a neural semantic parsing approach which models semantic par- sing as an end-to-end semantic graph generation process. |
| Outcome: | The proposed model achieves state-of-the-art performance on Overnight dataset and gets competitive performance on Geo and Atis datasets. |
Fine-grained Contrastive Learning for Definition Generation (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent pre-trained transformer-based definition generation models lack effective representation learning to contain full semantic components of the given word, leading to under-specific definitions. |
| Approach: | They propose a novel contrastive learning method that encourages the model to capture more detailed semantic representations from the definition sequence encoding. |
| Outcome: | The proposed method could generate more specific definitions compared with state-of-the-art models. |
MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing work on math word problem solvers replace real numbers with symbolic placeholders to focus on logic reasoning. |
| Approach: | They propose to inject numerical properties into symbolic placeholders with contextualized representation learning schema to solve number representation dilemma. |
| Outcome: | The proposed model can solve MWP problems on English and Chinese benchmarks. |
Improving Unsupervised Out-of-domain detection through Pseudo Labeling and Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Unsupervised OOD detection is a task aimed at discriminating whether given samples are from the in-domain (IND) . previous studies adopted the one-class classification approach, assuming that the training samples come from a single domain. |
| Approach: | They propose a framework that leverages latent categorical information to improve representation learning for textual OOD detection. |
| Outcome: | The proposed framework significantly outperforms baseline models on three datasets. |
LangSAMP: Language-Script Aware Multilingual Pretraining (2025.acl-long)
Copied to clipboard
| Challenge: | Recent multilingual pretrained language models often avoid using language embeddings, which places a significant burden on token representations to encode all language-specific information. |
| Approach: | They propose a method that incorporates both language and script embeddings into the output of Transformer blocks before passing the final representations to the language modeling head for prediction. |
| Outcome: | The proposed method outperforms the baseline model in zero-shot crosslingual transfer across diverse downstream tasks. |
Morphology-Aware Multi-Granularity Representation Learning for Agglutinative Languages (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for learning low-resource agglutinative languages are limited to word and phrase levels. |
| Approach: | They propose a morphology-aware gated multi-granularity pre-training framework for agglutinative languages . framework leverages morphological knowledge and integrates a word-level encoder to capture contextual semantics . |
| Outcome: | The proposed framework improves on Mongolian and Turkish agglutinative languages . it leverages morphological knowledge and integrates tagging and segmentation to build fine-grained representations . |
Word Representation Learning in Multimodal Pre-Trained Transformers: An Intrinsic Evaluation (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing models for linguistic representations of words are based on information extracted from large text corpora, and the sensory-motor experiences humans have with the world play an important role in determining word meaning. |
| Approach: | They propose to use contextualized word representations to learn semantic representations of words that align with human semantic intuitions. |
| Outcome: | The proposed models are shown to be more efficient on concrete word pairs than on abstract ones. |
MICO: A Multi-alternative Contrastive Learning Framework for Commonsense Knowledge Representation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to commonsense reasoning include fine-tuning large pre-trained language models or injecting the entire knowledge base for CKGC. |
| Approach: | They propose to learn commonsense knowledge representation by using a multi-alternative contrastive learning framework on COmmonsense Knowledge graphs. |
| Outcome: | Extensive experiments show that the proposed framework is effective in commonsense reasoning tasks. |
Watch the Neighbors: A Unified K-Nearest Neighbor Contrastive Learning Framework for OOD Intent Discovery (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for finding out-of-domain intents suffer from in-domain overfitting problem . previous methods fail to transfer prior knowledge to downstream clustering . |
| Approach: | They propose a unified K-nearest neighbor contrastive learning framework to discover OOD intents . they propose IND pre-training objective to learn discriminative features while maintaining intra-class diversity . |
| Outcome: | The proposed framework improves on three benchmark datasets. |
Retrieved Sequence Augmentation for Protein Representation Learning (2024.emnlp-main)
Copied to clipboard
Chang Ma, Haiteng Zhao, Lin Zheng, Jiayi Xin, Qintong Li, Lijun Wu, Zhihong Deng, Yang Lu, Qi Liu, Sheng Wang, Lingpeng Kong
| Challenge: | Using multiple sequence alignments (MSA) to extract evolutionary knowledge is limited. |
| Approach: | They propose to use multiple sequence alignments to augment protein representations . they propose to employ Retrieved Sequence Augmentation to enhance protein representation learning . |
| Outcome: | The proposed method surpasses MSA Transformer by 5% in structural and property prediction tasks while being 373 times faster. |
Enhancing Discriminative Representation in Similar Relation Clusters for Few-Shot Continual Relation Extraction (2025.naacl-long)
Copied to clipboard
Anh Duc Le, Nam Le Hai, Thanh Xuan Nguyen, Linh Ngo Van, Nguyen Thi Ngoc Diep, Sang Dinh, Thien Huu Nguyen
| Challenge: | Existing methods for relation extraction (RE) fail to address the problem of similar relations, which contributes to catastrophic forgetting. |
| Approach: | They propose a relation extraction method that utilizes relation descriptions and dynamic clustering to identify similar relations. |
| Outcome: | The proposed method mitigates catastrophic forgetting and outperforms state-of-the-art methods by a large margin. |
Intent Detection and Discovery from User Logs via Deep Semi-Supervised Contrastive Clustering (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to intent detection rely on epoch wise clustering and classification based on labeled and unlabeled data. |
| Approach: | They propose an end-to-end deep contrastive clustering algorithm that jointly updates model parameters and cluster centers via supervised and self-supervised learning. |
| Outcome: | The proposed approach outperforms baselines on five public datasets and human-in-the-loop variant for practical deployment. |
Multi-document Summarization with Maximal Marginal Relevance-guided Reinforcement Learning (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on single-document summarization (SDS) benefit from advances in neural sequence learning, but they produce unsatisfactory results on multi-document summary (MDS). |
| Approach: | They propose a neural sequence learning method that unifies advanced neural SDS methods and statistical measures used in classical MDS. |
| Outcome: | The proposed method achieves state-of-the-art performance on benchmark MDS datasets. |
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to model coarse-grained linguistic information do not integrate coarse-gram information into pre-training. |
| Approach: | They propose an explicitly n-gram masking method to enhance integration of coarse-grained linguistic information into pre-training. |
| Outcome: | The proposed method outperforms existing models on English and Chinese text corpora and fine-tunes on 19 downstream tasks. |
GraphDialog: Integrating Graph Knowledge into End-to-End Task-Oriented Dialogue Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | End-to-end task-oriented dialogue systems aim to generate system responses directly from plain text inputs. |
| Approach: | They propose a recurrent cell architecture which exploits the structural information in dialogue history . they propose recursive cell architecture to allow representation learning on graphs . |
| Outcome: | The proposed model improves on two different datasets on task-oriented dialogues. |
Learning to Disentangle Interleaved Conversational Threads with a Siamese Hierarchical Network and Similarity Ranking (N18-1)
Copied to clipboard
| Challenge: | Existing methods to disentangle interleaved conversations can lead to difficulties in following discussions and retrieving relevant information from simultaneous messages. |
| Approach: | They propose to leverage representation learning to separate intermingled messages into detached conversations by estimating conversation-level similarity between closely posted messages. |
| Outcome: | The proposed approach outperforms baselines in pairwise similarity estimation and conversation disentanglement. |
Better Pre-Training by Reducing Representation Confusion (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to improve pre-trained language models address information confusion in position encoding and model representations. |
| Approach: | They propose two techniques to improve pre-trained language models by decoupling directions and auxiliary regularizers. |
| Outcome: | The proposed techniques can improve pre-trained language models on GLUE benchmarks. |
ExCAR: Event Graph Knowledge Enhanced Explainable Causal Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work infers the causation between events based on knowledge from annotated causal event pairs, but additional evidence information is unexploited. |
| Approach: | They propose an Event graph knowledge enhanced explainable CAusal Reasoning framework that acquires additional evidence information from a large-scale causal event graph as logical rules for causal reasoning. |
| Outcome: | The proposed framework outperforms state-of-the-art methods in human evaluation and in animal models. |
Data-to-text Generation with Entity Modeling (P19-1)
Copied to clipboard
| Challenge: | Recent approaches to data-to-text generation have shown great promise thanks to the use of large-scale datasets and the application of neural network architectures which are trained end-to end. |
| Approach: | They propose an entity-centric neural architecture for data-to-text generation which uses hierarchical attention to create entity-specific representations which are dynamically updated. |
| Outcome: | The proposed model outperforms baselines in automatic and human evaluation on the RotoWire benchmark and a five-times larger dataset on the baseball domain. |
An Exploration of Three Lightly-supervised Representation Learning Approaches for Named Entity Classification (C18-1)
Copied to clipboard
| Challenge: | a recent study compares semi-supervised learning methods with bootstrapping methods . semi-semi-supervised methods reduce the amount of semantic drift introduced by iterative approaches . |
| Approach: | They propose to adapt three semi-supervised representation learning methods to an information extraction task . they show that all methods outperform state-of-the-art semi-representation learning methods . |
| Outcome: | The proposed methods outperform state-of-the-art semi-supervised methods on named entity classification task. |
Enhancing Low-resource Fine-grained Named Entity Recognition by Leveraging Coarse-grained Datasets (2023.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) often suffers from insufficient labeled data when the number of annotations exceeds several tens of labels. |
| Approach: | They propose a model with a fine-to- coarse mapping matrix to leverage hierarchical structure explicitly. |
| Outcome: | The proposed model outperforms both K-shot learning and supervised learning methods when dealing with a small number of fine-grained annotations. |
Enhancing Dual-Encoders with Question and Answer Cross-Embeddings for Answer Retrieval (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to solve question answering (QA) problems are limited by the need for text generation and answer retrieval. |
| Approach: | They propose to introduce QA interaction features in scoring function but at the cost of low efficiency in inference stage. |
| Outcome: | The proposed framework significantly outperforms the state-of-the-art method on multiple answer retrieval datasets. |
Do Trajectories Encode Verb Meaning? (2022.naacl-main)
Copied to clipboard
| Challenge: | Distributional models learn representations of words from text but lack grounding or the linking of text to the non-linguistic world. |
| Approach: | They investigate the extent to which trajectories naturally encode verb semantics . they build a procedurally generated agent-object-interaction dataset and compare methods . |
| Outcome: | The proposed model can capture verb semantics by tracing trajectories and self-supervised pretraining. |
Learning Neural Representation for CLIR with Adversarial Framework (D18-1)
Copied to clipboard
| Challenge: | Existing studies in cross-language information retrieval (CLIR) use general text representation models that are not optimized for the target task. |
| Approach: | They propose a novel text representation model based on adversarial learning which seeks a task-specific embedding space for CLIR. |
| Outcome: | The proposed model outperforms state-of-the-art continuous space models and is better than the strong machine translation baseline. |
Deep Probabilistic Logic: A Unifying Framework for Indirect Supervision (D18-1)
Copied to clipboard
| Challenge: | Indirect supervision is a promising direction to address the annotation bottleneck . end-to-end modeling with probabilistic logic is often intractable due to inference and learning . |
| Approach: | They propose a framework for indirect supervision that integrates deep learning with deep learning by combining probabilistic logic with deep-learning. |
| Outcome: | Experiments on biomedical machine reading demonstrate the potential of this framework. |
Improving Self-training for Cross-lingual Named Entity Recognition with Contrastive and Prototype Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to bridge the linguistic gap between self-training and monolingual named entity recognition (NER) however, due to sub-optimal performance on target languages, the pseudo labels are noisy and limit the overall performance. |
| Approach: | They propose to combine representation learning and pseudo label refinement in one coherent framework to improve self-training for cross-lingual named entity recognition (NER) |
| Outcome: | The proposed method improves cross-lingual named entity recognition (NER) on multiple transfer pairs. |
Bi-DCSpell: A Bi-directional Detector-Corrector Interactive Framework for Chinese Spelling Check (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Chinese Spelling Check (CSC) aims to detect and correct potentially misspelled characters in Chinese sentences. |
| Approach: | They propose a bi-directional Detector-Corrector framework for Chinese Spelling Check which mutually enhances the feature representation for detection and correction subtasks. |
| Outcome: | The proposed framework reduces the risk of over-correction and under-corrections while preserving the knowledge learnt from correction. |
Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation (2024.findings-acl)
Copied to clipboard
| Challenge: | Advancing representation learning in specialized fields like medicine remains challenging due to the scarcity of expert annotations for text and images. |
| Approach: | They propose a Fact Extractor that leverages large language models to extract factual statements from radiology reports. |
| Outcome: | The proposed framework outperforms current state-of-the-art methods in sentence ranking, natural language inference, and label extraction tasks. |
Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to learn the transfer from speech to text are unexplored . how to solve the representation discrepancy of speech and text is unexplorable . |
| Approach: | They propose a cooperative acoustic and linguistic representation learning method to fuse and utilize contextual information of speech and text. |
| Outcome: | The proposed method outperforms existing methods on low-resource speech recognition. |
Multilingual Molecular Representation Learning via Contrastive Pre-training (2022.acl-long)
Copied to clipboard
| Challenge: | Molecular representation learning is an essential role in cheminformatics, but language model-based approaches focus on local features, hence they may not capture global information. |
| Approach: | They propose a multilingual molecular embedding generation approach that uses two different languages to train a given molecule. |
| Outcome: | The proposed approach is pre-trained using SMILES and IUPAC as two different languages on large-scale molecules. |
Crisscrossed Captions: Extended Intramodal and Intermodal Semantic Similarity Judgments for MS-COCO (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing image captioning datasets have limited cross-modal associations, preventing researchers from examining how inter-modal learning impacts intra-modal tasks. |
| Approach: | They propose to use image captioning data to support multi-modal retrieval training and evaluation to assess the impact of inter-modality learning. |
| Outcome: | The proposed model is able to measure the influence of intra- and inter-modality learning. |
Answer-state Recurrent Relational Network (AsRRN) for Constructed Response Assessment and Feedback Grouping (2023.findings-emnlp)
Copied to clipboard
| Challenge: | STEM educators must trade off the ease of assessing selected response (SR) questions with constructed response (CR) questions, where students articulate their own reasoning. |
| Approach: | They propose to use multiple choice questions to construct response questions . they use a relational neural network to relate context, questions, reference responses, and students’ answers to a new dataset of 6,532 student responses to three, two-part CR questions. |
| Outcome: | The proposed network outperforms classifiers based on LLMs and few-shot learning with GPT-3.5 on a new dataset of 6,532 student responses to three, two-part CR questions. |
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale (D19-1)
Copied to clipboard
| Challenge: | Recent advances in representation learning have separated progress in both IR and MC . few studies have examined the relationship between retrieval and comprehension at different levels of granularity for development of MRS systems. |
| Approach: | They propose a simple yet effective pipeline system with consideration on hierarchical semantic retrieval at both paragraph and sentence level and their potential effects on the downstream task. |
| Outcome: | The proposed system achieves state-of-the-art on the leaderboard test sets of both FEVER and HOTPOTQA. |
HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models for hierarchical text classification do not consider statistical constraint on label representations learned by structure encoder. |
| Approach: | They propose a new hierarchical text classification model called HTCInfoMax which incorporates two modules to improve the model's representations. |
| Outcome: | The proposed model can model the interaction between each text sample and its ground truth labels explicitly which filters out irrelevant information. |
Supervised Neural Clustering via Latent Structured Output Learning: Application to Question Intents (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work on structured prediction has produced very effective supervised clustering algorithms using linear classifiers. |
| Approach: | They propose to use latent structured prediction loss and Transformer models to approach supervised clustering. |
| Outcome: | The proposed approach outperforms the state-of-the-art in recreating intents from public question corpora. |
MingOfficial: A Ming Official Career Dataset and a Historical Context-Aware Representation Learning Framework (2023.emnlp-main)
Copied to clipboard
You-Jun Chen, Hsin-Yi Hsieh, Yu Lin, Yingtao Tian, Bert Chan, Yu-Sin Liu, Yi-Hsuan Lin, Richard Tsai
| Challenge: | In Chinese studies, understanding the nuanced traits of historical figures can be challenging due to the need for domain expertise, specialist knowledge, and context-specific insights. |
| Approach: | They propose a large-scale multi-modal dataset for Chinese officials from the Ming Dynasty that integrates structured and text data to enable investigation of social structures. |
| Outcome: | The proposed dataset could enable exploratory analysis of official identities and significantly boost performance in tasks such as identifying nuance identities from 24.6% to 98.2% F1 score in hold-out test set. |
Beyond the Scientific Document: A Citation-Aware Multi-Granular Summarization Approach with Heterogeneous Graphs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results demonstrate that our model outperforms existing approaches for summarizing documents. |
| Approach: | proposed model constructs a heterogeneous graph to represent a document and its relevant external citations. |
| Outcome: | The proposed model outperforms existing models in three different scenarios. |
RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogue systems generate accurate and informative responses, but they are prone to hallucination problems. |
| Approach: | They propose a method to generate hallucinated responses using knowledge graphs . they propose local knowledge grounding to combine textual embeddings with corresponding KG embeddments . a global knowledge ground technique is also proposed to equip RHO with multi-hop reasoning abilities . |
| Outcome: | The proposed approach outperforms state-of-the-art methods on automatic and human evaluation by a large margin. |
Sequential Path Signature Networks for Personalised Longitudinal Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Current work on low-dimensional static user representations or more importantly on dynamic user representation is limited. |
| Approach: | They propose to integrate path signatures from rough path theory into neural sequential models by integrating contextual neural representations and recursive neural networks. |
| Outcome: | The proposed model outperforms state-of-the-art models on macro-average F1 score on two available datasets and outperformed previous models which only have access to historical posts. |
Joint Extraction of Entities, Relations, and Events via Modeling Inter-Instance and Inter-Label Dependencies (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing models that perform information extraction tasks manually assume heuristic dependency between the task instances and mean-field factorization for the joint distribution of instance labels. |
| Approach: | They propose to induce a dependency graph among task instances to boost representation learning by estimating their joint distribution via Conditional Random Fields. |
| Outcome: | The proposed model outperforms previous models on multiple IE tasks across 5 datasets and 2 languages. |
Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent discrete latent variable models have received a surge of interest in both NLP and CV . they are comparable to the continuous counterparts in representation learning, but are more interpretable in their predictions. |
| Approach: | They develop a topic-informed discrete latent variable model for semantic textual similarity . they inject the quantized representation into a transformer-based language model . |
| Outcome: | The proposed model outperforms strong baselines in semantic textual similarity tasks. |
Disentangling Categorization in Multi-agent Emergent Communication (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent work on the emergence of language between artificial agents has not isolated the effect of categorization power on inter-communication ability. |
| Approach: | They propose to use disentangled representations to quantify categorization power of agents to enable differential analysis between combinations of heterogeneous systems. |
| Outcome: | The proposed method reduces signaling accuracy by 40% despite encouraging compositionality in the artificial language. |
Shared-Private Bilingual Word Embeddings for Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Word embedding is central to neural machine translation, but indirectly interfaces with other layers, making them comparatively isolated. |
| Approach: | They propose a shared-private bilingual word embedding which gives a closer relationship between the source and target embedders and reduces the number of model parameters. |
| Outcome: | The proposed model improves on 5 language pairs belonging to 6 different language families and written in 5 different alphabets and significantly reduces model parameters. |
HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing taxonomies have limited coverage due to expensive manual curation process. |
| Approach: | They propose an algorithm that expands existing taxonomies to preserve their structure in a more expressive hyperbolic embedding space and learns to represent concepts and their relations with a hyperbolical Graph Neural Network. |
| Outcome: | The proposed algorithm outperforms baseline models with representation learning in a Euclidean feature space and achieves state-of-the-art performance on the taxonomy expansion benchmarks. |
Task-Oriented Clustering for Dialogues (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for task-oriented dialogue clustering are difficult to apply directly due to inherent differences between them. |
| Approach: | They propose a Dialogue Task Clustering Network model for task-oriented clustering . they use context-aware utterance representations and cross-dialogue utterrance cluster representations . |
| Outcome: | The proposed model outperforms baselines on three public datasets on all metrics. |
A Surprisingly Effective Fix for Deep Latent Variable Modeling of Text (D19-1)
Copied to clipboard
| Challenge: | Variational Autoencoders are powerful language models and effective representation learning frameworks. |
| Approach: | They propose a fix for posterior collapse which improves held-out likelihood, reconstruction and latent representation learning . |
| Outcome: | The proposed fix significantly improves held-out likelihood, reconstruction, and latent representation learning compared with previous state-of-the-art methods. |
Exploiting Document Structures and Cluster Consistencies for Event Coreference Resolution (2021.acl-long)
Copied to clipboard
| Challenge: | Existing deep learning models for event coreference resolution are limited in that they cannot exploit important interactions between relevant objects for ECR. |
| Approach: | They propose a deep learning model that groups coreferent event mentions into the same clusters . they use document structures to capture relevant objects for ECR . |
| Outcome: | The proposed model achieves state-of-the-art on two benchmark datasets. |
Multi-view Models for Political Ideology Detection of News Articles (D18-1)
Copied to clipboard
| Challenge: | Existing models for automatic detection of political ideology only leverage textual cues to identify the ideology evinced by a news article. |
| Approach: | They propose a novel attention based multi-view model that leverages cues from textual content and the network structure of news articles to identify political ideology. |
| Outcome: | The proposed model outperforms state of the art models by 10 percentage points on a battery of baselines and compares with baselines. |
Taking Actions Separately: A Bidirectionally-Adaptive Transfer Learning Method for Low-Resource Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to train NMT models rely on sparse parallel data . a variety of PC variants yield significant improvements for low-resource NMT . |
| Approach: | They propose to transfer well-trained NMT models to low-resource languages by bidirectionally-adaptive learning strategy . they divide inner constituents of Parent encoder into two "teams" aiming to adapt to characteristics of low- and high-resourced languages . |
| Outcome: | The proposed method improves on low-resource NMT models with a variety of PC variants. |
Reasoning over Entity-Action-Location Graph for Procedural Text Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Procedural text understanding aims at tracking the states and locations of entities mentioned in a paragraph. |
| Approach: | They propose a framework to model entities-entity, action, and location relations using a graph neural network. |
| Outcome: | The proposed approach outperforms strong baselines on two datasets, ProPara and Recipes. |
From Text Segmentation to Enhanced Representation Learning: A Novel Approach to Multi-Label Classification for Long Texts (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models rely on pre-trained language models, which have a maximum input sequence length of 512 tokens, and therefore have 'input length limitation'. |
| Approach: | They propose a text segmentation algorithm which guarantees to produce the optimal segmentation to address the issue of input length limitation caused by PLMs. |
| Outcome: | The proposed method improves both text and label representations on MLTC datasets, unraveling the intricate correlations between texts and labels. |
RotateQVS: Representing Temporal Information as Rotations in Quaternion Vector Space for Temporal Knowledge Graph Completion (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for temporal knowledge graphs can hardly model temporal relation patterns, lacking of interpretability. |
| Approach: | They propose a temporal modeling method which represents temporal entities as Rotations in Quaternion Vector Space and relations as complex vectors in Hamilton’s quaterniont space. |
| Outcome: | The proposed method can model key patterns of relations in TKG, such as symmetry, asymmetry, and inverse, and can capture time-evolved relations by theory. |
Multilingual SubEvent Relation Extraction: A Novel Dataset and Structure Induction Method (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for subevent relation extraction (SRE) focus on sequential order of words in texts to enhance representation learning. |
| Approach: | They propose a method that learns to induce effective graph structures for input texts . they use word alignment frameworks with dependency paths and optimal transport . |
| Outcome: | The proposed method is able to induce effective graph structures for input texts to boost representation learning. |
Compositionality and Generalization In Emergent Languages (2020.acl-main)
Copied to clipboard
| Challenge: | a new study examines whether emergent languages possess compositionality . compositionality is a core concept in linguistics, but linguists' definitions assume full knowledge of primitive expressions and their combination rules. |
| Approach: | They propose to use compositionality to combine expressions according to systematic rules to refer to composite concepts. |
| Outcome: | The proposed language has compositionality, but it is not generalized, the authors show . they show that the more compositional a language is, the more easily it will be picked up by new learners . |
The Dots Have Their Values: Exploiting the Node-Edge Connections in Graph-based Neural Models for Document-level Relation Extraction (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction do not capture the representations of the nodes in the graphs. |
| Approach: | They propose to explicitly compute the representations for the nodes in the graph-based edge-oriented model for Document-level Relation Extraction (DRE) . they propose to introduce two novel representation regularization mechanisms to improve the representation vectors for DRE. |
| Outcome: | The proposed model achieves state-of-the-art performance on two benchmark datasets. |
Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning have enabled several approaches to successfully parse more complex queries, but these models require a large amount of annotated training data to parser on new domains (e.g. reminder, music). |
| Approach: | They propose a method that adapts task-oriented semantic parsers to low-resource domains and outperforms a supervised neural model at a 10-fold data reduction. |
| Outcome: | The proposed method outperforms baseline methods on a newly collected multi-domain task-oriented semantic parsing dataset (TOPv2) . |
Modeling Document-Level Context for Event Detection via Important Context Selection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for Event Detection (ED) do not encode long-range document-level context . e.g., BERT cannot encode long text-level contextual information . |
| Approach: | They propose a method to model document-level context for Event Detection using transformer-based language models. |
| Outcome: | The proposed model can predict event prediction of target sentence in document-level context . the proposed model is effective on multiple benchmark datasets . |
Crosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to crosslingual Relation and Event Extraction (REE) suffer from monolingual bias due to training of models on source language data. |
| Approach: | They propose to use unlabeled data in target language to aid alignment of crosslingual representations by fooling a language discriminator. |
| Outcome: | The proposed method significantly advances the state-of-the-art in crosslingual REE tasks. |
Improving Event Coreference Resolution Using Document-level and Topic-level Information (2022.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that our model outperforms the SOTA baselines due to the encoding length limitation. |
| Approach: | They propose a longformer-based encoder and an encoder with a trigger-mask mechanism to learn sentence-level embeddings based on local context. |
| Outcome: | The proposed model outperforms the baselines on the KBP 2017 dataset. |
Content- and Topology-Aware Representation Learning for Scientific Multi-Literature (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches focus on learning textual information at sentence- or document-level, but ignore inter-document connections. |
| Approach: | They propose a model that extends representation learning to the multi-document level . it integrates latent semantic and rich relatedness information from topological networks . |
| Outcome: | The proposed model learns latent semantic information from content and rich relatedness information from topological networks. |
Capturing Regional Variation with Distributed Place Representations and Geographic Retrofitting (D18-1)
Copied to clipboard
| Challenge: | Dialects are one of the main drivers of language variation, a major challenge for natural language processing tools. |
| Approach: | They use a corpus of 16.8M anonymous online posts to learn continuous document representations of cities. |
| Outcome: | The proposed method matches dialect areas at different granularities against an existing dialect map. |
StrAE: Autoencoding for Pre-Trained Embeddings using Explicit Structure (2023.emnlp-main)
Copied to clipboard
| Challenge: | Structured Autoencoder framework StrAE enables effective learning of multi-level representations through strict adherence to explicit structure. |
| Approach: | They propose a Structured Autoencoder framework that strictly adheres to explicit structure and uses a contrastive objective over tree-structured representations. |
| Outcome: | The proposed framework outperforms baselines that don’t involve explicit hierarchical compositions and is comparable to models given informative structure. |
KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundi (2020.coling-main)
Copied to clipboard
| Challenge: | low-resource African languages are traditionally left behind because of the lack of well-annotated data and effective preprocessing. |
| Approach: | They propose two news datasets for multi-class classification of news articles in two low-resource African languages. |
| Outcome: | The proposed datasets show that training embeddings on the higher-resourced Kinyarwanda yields successful cross-lingual transfer to Kirundi. |
Reasoning-Based Refinement of Unsupervised Text Clusters with LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Unsupervised methods are used to induce latent semantic structure from large text collections, yet their outputs often contain incoherent, redundant, or poorly grounded clusters. |
| Approach: | They propose a framework that leverages large language models as semantic judges to validate and restructure unsupervised clustering algorithms. |
| Outcome: | The proposed framework improves cluster coherence and human-aligned labeling quality over traditional models and representation-based baselines. |
Knowledge Base Embedding By Cooperative Knowledge Distillation (2020.coling-main)
Copied to clipboard
| Challenge: | Knowledge bases are increasingly exploited as gold standard data sources for various knowledge-driven NLP tasks. |
| Approach: | They propose a method to perform knowledge base representation learning by mutually and jointly distilling knowledge within a dynamic teacher-student setting. |
| Outcome: | The proposed approach outperforms two baselines, traditional and sequential, on two standard datasets showing that it is possible to distill knowledge between KBs. |
UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect out-of-domain (OOD) intents ignore alignment between representation learning and scoring function, limiting performance. |
| Approach: | They propose a unified neighborhood learning framework to detect OOD intents . they propose to align representation learning with scoring function . |
| Outcome: | The proposed method is able to detect out-of-domain (OOD) intents from user queries. |
Does Corpus Quality Really Matter for Low-Resource Languages? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on multilingual pre-training has relied on automatically filtered versions of CommonCrawl. |
| Approach: | They propose to use tailored crawling to identify and scrape websites with high-quality content to improve representation learning in Basque. |
| Outcome: | The proposed corpus, called EusCrawl, has a much higher quality according to native annotators than the Basque portion of popular multilingual corpora like CC100 and mC4. |
Similarity-Based Reconstruction Loss for Meaning Representation (D18-1)
Copied to clipboard
| Challenge: | a new paper proposes and evaluates a set of loss functions that can be used to train models for representation learning . cross-entropy loss penalizes models when they fail to generate the exact word from ground truth data . |
| Approach: | They propose and evaluate loss functions that can be used to train any neural model for representation learning. |
| Outcome: | The proposed loss functions amplify semantic diversity while preserving original meaning . they show performance improvement on paraphrase detection and language inference tasks . |
On Efficient Retrieval of Top Similarity Vectors (D19-1)
Copied to clipboard
| Challenge: | Existing representation learning methods such as Word2vec represent word embeddings in the semantic space. |
| Approach: | They propose an efficient method for searching vectors via a non-metric matching function: inner product. |
| Outcome: | Experiments on data representations learned for different machine learning tasks show the proposed method outperforms existing methods. |
Decoupling Pseudo Label Disambiguation and Representation Learning for Generalized Intent Discovery (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for generalized intent discovery lack pseudo label disambiguation and representation learning. |
| Approach: | They propose a prototype learning framework to decouple pseudo label disambiguation and representation learning. |
| Outcome: | The proposed method can decouple pseudo label disambiguation and representation learning. |
KV-Embedding: Training-free Text Embedding via Internal KV Re-routing in Decoder-only LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work shows that decoder-only LLMs can serve as strong embedding backbones when fine-tuned with contrastive objectives. |
| Approach: | They propose a framework that activates the latent representation power of frozen LLMs by rerouting the final token's KV states as a prepended prefix. |
| Outcome: | The proposed framework outperforms existing training-free baselines by 10% on MTEB and maintains robust performance on sequences up to 4,096 tokens. |
Aspect-Level Sentiment Analysis Via Convolution over Dependency Tree (D19-1)
Copied to clipboard
| Challenge: | Existing methods to identify sentiment polarity of opinion words are cumbersome due to the amount of opinionated material on the internet. |
| Approach: | They propose a method to identify sentiment polarity of opinion words on a specific aspect of a sentence using neural networks. |
| Outcome: | The proposed method is the state-of-the-art in aspect-based sentiment classification. |
Learning Disentangled Representations of Negation and Uncertainty (2022.acl-long)
Copied to clipboard
| Challenge: | Negation and uncertainty modeling are long-standing tasks in natural language processing. |
| Approach: | They propose to disentangle negation, uncertainty, and content using a Variational Autoencoder by supervising latent representations using auxiliary objectives. |
| Outcome: | The proposed model can disentangle negation, uncertainty, and content using a Variational Autoencoder. |
Describe Me an Auklet: Generating Grounded Perceptual Category Descriptions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Learning and using abstract perceptual concepts is under-investigated in the language-and-vision field. |
| Approach: | They propose a framework for testing category-level perceptual grounding in multi-modal language models by using separate neural networks to generate and interpret descriptions of visual categories. |
| Outcome: | The proposed framework compares prototype- and interpretation-based representations with the performance of the generation model and the interpretation model, which is an indicator of perceptual grounding. |
Representation Learning for Information Extraction from Form-like Documents (2020.acl-main)
Copied to clipboard
| Challenge: | Form-like documents like invoices, purchase orders, tax forms and insurance quotes are common in day-to-day business workflows, but current techniques for processing them largely still employ manual effort or brittle and error-prone heuristics for extraction. |
| Approach: | They propose an extraction system that uses knowledge of the types of the target fields to generate extraction candidates and a neural network architecture that learns a dense representation of each candidate based on neighboring words in the document. |
| Outcome: | The proposed system generates extraction candidates based on neighboring words in the document and is interpretable, as shown using loss cases. |
Representation Learning for Unseen Words by Bridging Subwords to Semantic Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained word embeddings only include words that appeared in corpora where pre-tried embedds are learned. |
| Approach: | They propose a method to represent out-of-vocabulary words using subword information and knowledge. |
| Outcome: | The proposed method improves performance over baselines that only use subwords or knowledge to represent OOV words. |
Dependency Graph Enhanced Dual-transformer Structure for Aspect-based Sentiment Classification (2020.acl-main)
Copied to clipboard
| Challenge: | Aspect-based sentiment classification is a popular task aimed at identifying the corresponding emotion of a given aspect. |
| Approach: | They propose a dependency graph enhanced dual-transformer network to support mutual reinforcement between the flat representation learning and graph-based representation learning. |
| Outcome: | The proposed model outperforms state-of-the-art methods on five datasets with a large margin. |
GreenKGC: A Lightweight Knowledge Graph Completion Method (2023.acl-long)
Copied to clipboard
| Challenge: | Knowledge graph completion (KGC) aims to discover missing relationships in knowledge graphs (KGs). |
| Approach: | They propose a modularized knowledge graph completion solution that learns embeddings for entities and relations through a score function. |
| Outcome: | Experimental results show that GreenKGC outperforms SOTA methods in low dimensions and even better against high-dimensional models with a much smaller model size. |
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding (2022.acl-long)
Copied to clipboard
Mengze Li, Tianbao Wang, Haoyu Zhang, Shengyu Zhang, Zhou Zhao, Jiaxu Miao, Wenqiao Zhang, Wenming Tan, Jin Wang, Peng Wang, Shiliang Pu, Fei Wu
| Challenge: | Existing methods for grounding video frames with dense annotations require enormous amount of human effort. |
| Approach: | They propose to ground natural language in video frames with only one frame labeled . they propose an end-to-end model that eliminates interference of irrelevant frames . |
| Outcome: | The proposed model can ground natural language in all video frames with only one frame labeled . the proposed model eliminates interference of irrelevant frames based on branch search and cropping techniques . |
A Spectral Viewpoint on Continual Relation Extraction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to solve the Continual Relation Extraction problem have been proposed . |
| Approach: | They propose a class-wise regularization method that preserves eigenvectors for each class shape . they propose spectral regularization to preserve eenvector shape after learning new tasks . |
| Outcome: | The proposed method improves performance on two benchmark datasets. |
MIR-GAN: Refining Frame-Level Modality-Invariant Representations with Adversarial Network for Audio-Visual Speech Recognition (2023.acl-long)
Copied to clipboard
| Challenge: | Audio-visual speech recognition (AVSR) leverages multimodal signals to understand human speech. |
| Approach: | They propose an adversarial network to refine frame-level modality-invariant representations to bridge the distribution gap between modalities. |
| Outcome: | The proposed approach outperforms the state-of-the-art on public benchmarks LRS3 and LRS2 on the modalities of AVSR. |
Revealing the Myth of Higher-Order Inference in Coreference Resolution (2020.emnlp-main)
Copied to clipboard
| Challenge: | Adapted coreference resolution models have only marginally improved performance over representation learning. |
| Approach: | They implement an end-to-end coreference system and four HOI approaches to analyze the impact of higher-order inference on coreference resolution. |
| Outcome: | The proposed model shows that the impact of higher-order inference (HOI) on coreference resolution is negative to marginal, providing a new perspective on the task. |
Learning First-Order Logic Rules for Argumentation Mining (2025.acl-long)
Copied to clipboard
Yang Sun, Guanrong Chen, Hamid Alinejad-Rokny, Jianzhu Bao, Yuqi Huang, Bin Liang, Kam-Fai Wong, Min Yang, Ruifeng Xu
| Challenge: | Argumentation Mining (AM) aims to extract argumentative structures from texts by identifying argumentation components (ACs) and their argumentative relations (ARs). |
| Approach: | They propose a First- Order Logic reasoning framework for AM to capture logical reasoning paths within argumentative texts. |
| Outcome: | The proposed framework outperforms strong baselines while significantly improving explainability. |
Feature Projection for Improved Text Classification (2020.acl-main)
Copied to clipboard
| Challenge: | In sentiment classification, there are some good features that are indicative of class labels, but there are also many common features that do not discriminate for classification. |
| Approach: | They propose to project existing features into the orthogonal space of the common features and make them more discriminative for classification. |
| Outcome: | The proposed method improves CNN, RNN, Transformer, and Bert based text classification and obtains markedly better results. |
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LMM-based embedding models exhibit a high degree of overlap in similarity distribution between positive and negative pairs, making it challenging to distinguish hard negative pairs effectively. |
| Approach: | They propose a framework that improves the embedding model's representation learning for negative pairs based on their discriminative difficulty. |
| Outcome: | The proposed framework improves the embedding model's representation learning for negative pairs based on their discriminative difficulty. |
SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval (2024.findings-acl)
Copied to clipboard
Siwei Wu, Yizhi Li, Kang Zhu, Ge Zhang, Yiming Liang, Kaijing Ma, Chenghao Xiao, Haoran Zhang, Bohao Yang, Wenhu Chen, Wenhao Huang, Noura Al Moubayed, Jie Fu, Chenghua Lin
| Challenge: | Multi-modal information retrieval (MMIR) is a rapidly evolving field . current benchmarks for image-text pairings overlook the scientific domain . |
| Approach: | They develop a scientific domain-specific MMIR benchmark to evaluate image-text pairings using open-access research paper corpora. |
| Outcome: | The proposed benchmarks are based on 530K image-text pairs extracted from scientific documents with detailed captions. |
Text-Attributed Knowledge Graph Enrichment with Large Language Models for Medical Concept Representation (2026.acl-long)
Copied to clipboard
| Challenge: | eHRs encode a patient's medical history as a high-dimensional and sparse sequence of diagnosis, medication, and procedure concepts . robust concept representation learning is hindered by key challenges, authors say . clinically important cross-type dependencies are often missing or incomplete in existing ontology resources . |
| Approach: | They propose a graph learning framework that integrates semantics with medical concepts to improve prediction performance. |
| Outcome: | The proposed framework improves prediction performance and integrates semantics with graph structure. |
DNA: Denoised Neighborhood Aggregation for Fine-grained Category Discovery (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn compact cluster representations from coarsely labeled data are noisy and degrade the quality of learning. |
| Approach: | They propose a framework that encodes semantic structures of data into the embedding space . they retrieve k-nearest neighbors of a query as positive keys to capture similarities . |
| Outcome: | The proposed framework can retrieve more accurate neighbors and outperform state-of-the-art models by a large margin. |
SamToNe: Improving Contrastive Loss for Dual Encoder Retrieval Models with Same Tower Negatives (2023.findings-acl)
Copied to clipboard
| Challenge: | Dual encoders have been used for retrieval tasks and representation learning with good results. |
| Approach: | They propose an improved contrastive learning objective by adding queries or documents from the same encoder towers to the negatives. |
| Outcome: | The proposed model improves retrieval quality for both symmetric and asymmetric dual encoders by adding queries or documents from the same encoder towers to the negatives. |
Rumor Detection on Twitter with Claim-Guided Hierarchical Graph Attention Networks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for rumor detection are limited to the strict relation of user responses or oversimplify the conversation structure. |
| Approach: | They propose a method that reinforces interaction of user opinions while reducing negative impact imposed by irrelevant posts. |
| Outcome: | The proposed method improves performance on three Twitter datasets and can detect rumors at early stages. |
Rethinking Negative Pairs in Code Search (2023.emnlp-main)
Copied to clipboard
| Challenge: | Comparative learning is a key component in fine-tuning code search models . however, negative samples of InfoNCE may deteriorate its representation learning . |
| Approach: | They propose a loss function that inserts weight terms into InfoNCE to improve contrastive learning. |
| Outcome: | The proposed loss function is a special case of Soft-InfoNCE, the authors show . it is more accurate than other loss functions, and it is faster than other models. |
PAR: Political Actor Representation Learning with Social Context and Expert Knowledge (2022.emnlp-main)
Copied to clipboard
Shangbin Feng, Zhaoxuan Tan, Zilong Chen, Ningnan Wang, Peisheng Yu, Qinghua Zheng, Xiaojun Chang, Minnan Luo
| Challenge: | Existing approaches focus on textual data and voting records to induce political actors' stances. |
| Approach: | They propose a Political Actor Representation learning framework that leverages social context and expert knowledge to model ideological stances. |
| Outcome: | The proposed framework improves political text understanding and improves roll call vote prediction and political perspective detection. |
Knowledge GeoGebra: Leveraging Geometry of Relation Embeddings in Knowledge Graph Completion (2024.lrec-main)
Copied to clipboard
| Challenge: | Knowledge graph embedding models are limited to the algebra and geometry of the entity embeddable space, the algebra of the relation embeddible space, and the interaction between relation and entity embeds. |
| Approach: | They propose a method that leverages the geometry of relation embeddings and generalizes it with the concept of a butterfly curve, consecutively. |
| Outcome: | The proposed model outperforms existing models on the WN18RR, FB15K-237 and YouTube benchmarks. |
ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis (2023.findings-acl)
Copied to clipboard
Yakun Yu, Mingjun Zhao, Shi-ang Qi, Feiran Sun, Baoxun Wang, Weidong Guo, Xiaoli Wang, Lei Yang, Di Niu
| Challenge: | Existing methods for multimodal sentiment analysis focus on general knowledge, which is inadequate to identify specific sentiments across modalities. |
| Approach: | They propose a method where specific-knowledge representations for each modality can be learned together with general knowledge representations via knowledge injection based on an adapter architecture. |
| Outcome: | The proposed method outperforms all prior methods on three popular benchmarks on multimodal sentiment analysis metrics. |
Intent Discovery with Frame-guided Semantic Regularization and Augmentation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing intent discovery methods focus on transferring prior knowledge of known intents to unknown ones. |
| Approach: | They propose to use frame knowledge as conceptual semantic guidance to bridge the gap between known intents representation learning and unknown intents clustering. |
| Outcome: | The proposed method outperforms solid baselines on two benchmark datasets. |
M3: A Multi-Task Mixed-Objective Learning Framework for Open-Domain Multi-Hop Dense Sentence Retrieval (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent research shows that contrastive learning can lead to suboptimal retrieval performance. |
| Approach: | They propose an advanced recursive Multi-hop dense sentence retrieval system built upon a novel Multi-task Mixed-objective approach for dense text representation learning. |
| Outcome: | The proposed approach yields state-of-the-art performance on a large-scale open-domain fact verification benchmark dataset, FEVER. |
IntenDD: A Unified Contrastive Learning Approach for Intent Detection and Discovery (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Intent-related tasks are typically modeled as separate tasks, but a unified approach is proposed . INTENDD uses an entirely unsupervised contrastive learning strategy for representation learning . |
| Approach: | They propose a unified approach to identifying intents from dialogue utterances . they propose an unsupervised contrastive learning strategy for representation learning . |
| Outcome: | The proposed approach outperforms baselines on three intent-related tasks on multiple datasets. |
See Detail Say Clear: Towards Brain CT Report Generation via Pathological Clue-driven Representation Learning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Brain CT report generation is important to aid physicians in diagnosing cranial diseases. |
| Approach: | They propose a Pathological Clue-driven Representation Learning model to build cross-modal representations based on pathological clues and adapt them for text generation. |
| Outcome: | The proposed method outperforms previous methods and achieves SoTA performance. |
TacoPrompt: A Collaborative Multi-Task Prompt Learning Method for Self-Supervised Taxonomy Completion (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for automating taxonomy completion use subtasks to learn subtask results, ignoring the effects of subtask on the final prediction. |
| Approach: | They propose a multi-task automatic taxonomy completion method that attaches emerging concepts to an appropriate pair of hypernym and hyponym in existing taxonomies. |
| Outcome: | The proposed method improves on three datasets and improves inference efficiency. |
Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods modify attention mechanism to be bidirectional, undermining LLMs’ ability to extract semantic information acquired during pre-training. |
| Approach: | They propose a general-purpose embedding model that pre-encodes input text into a single Contextual token and then prepends it to the LLM's input sequence. |
| Outcome: | The proposed model improves performance of decoder-only large language models without altering their architectures or introducing significant computational overhead. |
LAMCL: A Length-aware Momentum Contrastive Learning Framework for Multiscale Machine-Revised Text Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Recent detection methods struggle to capture fine-grained semantic differences, especially for short texts. |
| Approach: | They propose a framework for machine-revised text detection that integrates two modules to enhance discriminative semantic features. |
| Outcome: | The proposed method outperforms existing detectors in identifying machine-revised text across diverse practical scenarios, tasks, and LLMs. |
Beyond instruction-conditioning, MoTE: Mixture of Task Experts for Multi-task Embedding Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Instruction-conditioning has become the dominant approach for embedding specialization, but its direct application to low-capacity models imposes representational constraints that limit the performance gains derived from specialization. |
| Approach: | They propose a mixture of task experts transformer block which leverages task-specialized parameters trained with Task-Aware Contrastive Learning to enhance the model’s ability to generate specialized embeddings. |
| Outcome: | The proposed model achieves 64% higher performance gains in retrieval datasets (+3.27 +5.21) and 43% higher performance gain across all datasets (+1.81 2.60). |
Preserving Language Capabilities in Vision-Language Models via Representation Regulation (2026.findings-acl)
Copied to clipboard
| Challenge: | Vision-Language Models (VLMs) provide a unified framework to process both text-only and vision-language tasks. |
| Approach: | They propose a method to reduce the distance between visual and textual representations by introducing a Representation Distribution Difference (RDD) loss. |
| Outcome: | Empirical evidence shows that finetuning VLMs on vision-language data has degraded language capabilities. |
MERIT: Multi-Agent Collaboration for Unsupervised Time Series Representation Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to time series representation learning are time-consuming and expert-dependent, which are difficult to generalize across different tasks. |
| Approach: | They propose to use large language model agent to guide unsupervised time series representation learning and a framework to integrate three LLM agents to collaboratively generate positive views for time series data. |
| Outcome: | The proposed framework integrates large language model (LLM) agent to guide unsupervised time series representation learning and compares it with state-of-the-art baselines on multiple time series datasets. |
Scale-VAE: Preventing Posterior Collapse in Variational Autoencoder (2024.lrec-main)
Copied to clipboard
| Challenge: | Variational autoencoder (VAE) is a widely used generative model . but when employing strong autoregressive generation networks, VAE tends to converge to a degenerate local optimum known as posterior collapse. |
| Approach: | They propose a model called Scale-VAE to solve a posterior collapse problem . they use a factor to keep the posterior dimension discriminative across data instances . |
| Outcome: | The proposed model outperforms state-of-the-art models in density estimation and representation learning. |
STELLA: A Multimodal LLM for Protein Functional Annotation via Unified Sequence-Structure Encoding (2026.findings-acl)
Copied to clipboard
Hongwang Xiao, Wenjun Lin, Xi Chen, Hui Wang, Kai Chen, Jiashan Li, Yuancheng Sun, Sicheng Dai, Boya Wu, Qiwei Ye
| Challenge: | a multimodal protein language model (LLM) integrates sequence, structure, and function into functional annotation. |
| Approach: | They propose a multimodal protein language model that synergistically aligns bimodal representations with the textual modality to advance protein functional annotation. |
| Outcome: | The proposed model synergizes bimodal representations with the textual modality to advance protein functional annotation. |
Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingness (2025.emnlp-main)
Copied to clipboard
| Challenge: | Clinical notes are often missing from clinical notes, resulting in modality missing-not-at-random (MMNAR) . large language models fine-tuned or adapted to clinical tasks have shown promise in medical reasoning, outcome prediction, and decision support. |
| Approach: | They propose a framework that leverages observed data and informative missingness in multimodal clinical records. |
| Outcome: | The proposed framework leverages observed data and informative missingness in multimodal clinical records. |
Let Retrievers Think Before Action: Thought-Augmented Embedding for Dense Retrieval (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models have demonstrated that explicit step-by-step thinking can substantially improve performance on complex tasks. |
| Approach: | They propose a model that generates preliminary thoughts for input queries before document retrieval. |
| Outcome: | The proposed model generates preliminary thoughts for input queries before document retrieval. |